Accelerated Methods for Non-Convex Optimization

We present an accelerated gradient method for non-convex optimization problems with Lipschitz continuous first and second derivatives. The method requires time O( −7/4 log(1/ )) to find an -stationary point, meaning a point x such that ‖∇f(x)‖ ≤ . The method improves upon the O( −2) complexity of gradient descent and provides the additional second-order guarantee that ∇f(x) −O( )I for the computed x. Furthermore, our method is Hessian-free, i.e. it only requires gradient computations, and is therefore suitable for large scale applications.